Apparatus and method for converting a compressed video stream with preview of converted stream

ABSTRACT

An apparatus by which a user can easily set a parameter for conversion when a compression encoding system of video stream is converted is provided. A particular period in a first video stream is repeatedly reproduced to convert the reproduced portion into a second video stream in real time. At that time, the user checks the image quality after conversion on a preview screen and sets a desired conversion parameter. A local decode image is output to the preview screen. Thus, the circuit configuration is simplified.

RELATED APPLICATIONS

This is a continuation application of U.S. patent application Ser. No.10/441,012, filed on May 20, 2003, now allowed, the contents of whichare incorporated herein by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an apparatus for converting an encodingsystem of video stream or video transcoder, a method therefor, and avideo recording/reproducing device using the above apparatus.

2. Description of the Related Art

Technologies for digitizing image information and compression encodingby an MPEG video compression technology to store on a storage medium orto transmit through a communication medium have been used for varioustypes of equipment. Such examples are a DVD recording/reproducing deviceusing an MPEG-2 Video (ISO/IEC13818-2) system, communications betweenportable terminals using an MPEG-4 Video (ISO/IEC14496-2) system, andthe like. Because the compression systems are different depending onuses, each device needs to convert data between different compressionencoding systems.

To meet the above necessity, there is a device for transcoding the videostream compression coded by the MPEG-2 method into another compressionencoding system. At that time, a user desires to check the image qualityof the transcoded stream to select a compression parameter fortranscoding. And, a technology therefor has been proposed (e.g., referto JP-A-2001-36844).

SUMMARY OF THE INVENTION

JP-A-2001-36844 discloses to output the stream before transcoding andthe stream after transcoding simultaneously, decodes them and outputsthe decoded images at the same time. The user checks the images for adifference in image quality between them to select desired imagequality. Such a system reproduces the entire stream before theconversion and also operates to check the image quality of the convertedstream. Therefore, it is hard to set the image quality of a short streamportion in detail. For example, when the image quality of a relativelyshort stream such as moving image mail is to be determined, reproductionof the relevant portion ends so soon that it is difficult to conduct thesetting work.

There is also a demand for falling of a transmission time of theconverted video stream within a designated value when the converted datais sent through a network. There is also a demand for adjustment of aframe rate and an amount of codes allotted to the frame as the imagequality of each frame. When a bit rate is fixed, the frame rate and theallotted code amount of each frame are in a contrary relationship, sothat they must be set while being kept in balance. Users have demandedto make such detail setting interactively. But, the JP-A-2001-36844 hasnot fully considered such ease of use.

In view of the situations described above, the present inventionprovides a video transcoder by which a user can easily set a parameterat the time of conversion, a method for video transcoder, and a videorecording/reproducing device using it.

To achieve the object, the apparatus according to an aspect of thepresent invention comprises a reproducer means reproducing a first videostream recorded on a storage medium; a stream converter converting thereproduced first video stream into a second video stream; a conversionperiod designation means designating a conversion period in the firstvideo stream; a stream output unit outputting the converted second videostream; and a preview output unit outputting an image corresponding to adecoded image of the converted second video stream to a display device.The above stream output unit outputs the second video stream for aperiod designated by the conversion period designator.

Especially, the reproducer repeatedly reproduces the first video streamfor the period designated by the conversion period designator, and thepreview output unit repeatedly outputs an image corresponding to thedecoded image of the period designated by the conversion perioddesignator.

The apparatus according to another aspect of the present inventioncomprises a stream converter converting the first video stream into thesecond video stream; a stream output unit outputting the convertedsecond video stream; and a preview output unit outputting an imagecorresponding to a decoded image of the converted second video stream toa display device. The stream converter generates a local decode image ofthe second video stream for frame-to-frame motion compensation; and thepreview output unit outputs a local decode image generated by the streamconverter as an image corresponding to the decoded image of the secondvideo stream.

The apparatus according to the other aspect of the present inventionfurther comprises a parameter setting unit setting a parameter relatedto the conversion by the stream converter, wherein the stream outputunit starts outputting the second video stream after setting of aparameter of the stream converter by the parameter setting unit.

Accordingly, the user can readily set a parameter at the time ofconversion by checking the preview screen.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other objects, features and advantages of the presentinvention will become more readily apparent from the following detaileddescription when taken in conjunction with the accompanying drawings.

FIG. 1 is a block diagram showing a first embodiment of a videorecording/reproducing device including a video transcoder according tothe present invention;

FIG. 2 is a diagram showing an example of a preview screen of theembodiment shown in FIG. 1;

FIG. 3 is a diagram showing an example of an image signal shown on apreview screen;

FIG. 4 is a diagram showing another example of the image signal shown onthe preview screen;

FIG. 5 is a block diagram showing a second embodiment of the videorecording/reproducing device including a video transcoder according tothe present invention;

FIG. 6 is a block diagram showing a third embodiment of the videorecording/reproducing device including a video transcoder according tothe present invention;

FIG. 7 is a diagram illustrating that a conversion stream is generatedduring a designated period only;

FIG. 8 is a detail block diagram showing generation of a local decodeimage; and

FIG. 9 is a diagram illustrating a process of generating a preview imagefrom a local decode image.

DETAILED DESCRIPTION OF THE INVENTION

FIG. 1 is a block diagram showing the first embodiment of a videorecording/reproducing device containing a video transcoder forconverting an encoding system of video stream according to the presentinvention. This device records an imaging signal as an MPEG2 videostream in a storage medium, reproduces it to convert into an MPEG4 videostream and outputs to the outside. When a user wants to record on amedium 100, a recording command is output from a user IF 109, and asystem controller 110 having received the recording command controls alens section 400 and a digital signal processor (DSP) 401 to convert theimaged analog signal into a digital video signal. Besides, the digitalvideo signal is encoded by an MPEG2 encoder 402 to generate an MPEG2video stream. The MPEG2 video stream is recorded in the storage medium100 together with unshown audio information or video stream control datathrough a medium IF (record reproduction circuit) 101.

When the reproduction is instructed by the user IF 109, the MPEG2 videostream recorded in the medium 100 has a prescribed stream portionreproduced from the medium 100. The reproduced video stream is sent toan MPEG2 decoder 102 through the medium IF 101. The video stream decodedhere has generally an NTSC size image, and its effective pixel area hasa resolution of approximately 704 (horizontal)×480 (vertical) per frame.At the time of normal reproduction, the image is output with the abovesize to an outside display or the like.

Conversion of the reproduced MPEG2 stream into the MPEG4 stream will bedescribed. A scaler 103 for resolution conversion scales down the decodeimage of MPEG2 having the NTSC size to a resolution (e.g., approximatelyQCIF (176×144)) to be handled in MPEG4 and sends to an MPEG4 encoder104. The MPEG4 encoder 104 MPEG4-encodes the sent image and outputs tothe outside from an output terminal 105.

At that time, the MPEG4 encoder 104 stores the same image data as thatwhen each frame was decoded in a local decode frame memory 106 in orderto calculate motion compensation between the frames in the encodingprocess. Specifically, the image data stored in this local decode framememory 106 corresponds to an image which is actually obtained when thevideo stream MPEG4-encoded and output to the outside from the outputterminal 105 is decoded later. In this embodiment, the image data storedin the local decode frame memory 106 is used as image data for preview.A display circuit 107 reads the image data in display order from thelocal decode frame memory 106 and outputs it as a picture signal to thedisplay device from an output terminal 108. The user can observe thepreview image shown on the display device to check the image quality ofthe converted video stream to be output from the output terminal 105 tothe outside.

In this embodiment, a local decode image produced in the process ofencoding is used to generate the preview screen, so that it is notnecessary to include another decode circuit. It has an advantage thatthe circuit can be made small in size. In comparison with the case wherethe output stream is redecoded to form the preview screen, it also hasan advantage that a response time from the setting by the user to theoutput of an image reflecting the setting becomes short, and feedbackbecomes quick.

The display device has been described to be connected to the outside butmay be included in the main apparatus.

The generation of the local decode image described above will bedescribed in detail with reference to FIG. 8 and FIG. 9.

FIG. 8 shows a flow that the image which has the resolution converted bythe scaler 103 is encoded by the MPEG4 encoder 104 to generate a simpleprofile stream. For a P picture (predictive-coded picture), theresolution converted image is subject to motion search with thegenerated local decode image and the extraction of a frame-to-framedifference value by a motion compensation circuit in the MPEG4 encoder104. The extracted difference value is encoded to MPEG4 video stream byan encoding circuit 702. This stream is decoded by a local decoder 704to obtain a local decode image. This image is stored again in the localdecode frame memory 106 and referred to for encoding of a frame in thelater stage. For an I picture (intra-coded picture), motion compensationis not performed, and encoding is performed according to pixelinformation only in the frame.

FIG. 9 illustrates a process of generating a preview display image fromthe local decode frame by the display circuit 107. It is assumed thatthe image converted by the resolution conversion circuit 103 is suppliedat a frame rate of 30 fps (frame/sec). Among the resolution-convertedimages supplied at 30 fps, data on one image is extracted from everythree images to generate a local decode image. For example, the Ipicture is encoded and local decoded from the image of a frame 710according to the flow shown in FIG. 8 to generate an I frame 720. A Ppicture has motion compensation performed with reference to the picturebefore the resolution conversion and the previous local decode image.For example, a P frame 721 has motion compensation performed from aframe 713 after the resolution conversion and the I frame 720 of thelocal decode image. Then, a difference value is encoded and localdecoded to generate the local decode image 721. In this example, thelocal frame has a generation rate of 10 pfs, so that, for example inorder to output as NTSC video, the display circuit 107 repeatedlydisplays the local decode image to generate preview images 730 to 739.In the above configuration, the local decode frame has the same imagequality as that when the generated stream is decoded by an outsidedecoder, so that the image checked as a preview image reproduces adecoding situation faithfully.

FIG. 2 shows an example of the preview screen shown on the displaydevice in the embodiment of FIG. 1. When the preview operation isdesignated through the user IF 109, preview image data is output fromthe display circuit 107. The preview screen shows parameters of theimage quality, frame rate, bit rate and the like after the conversioninto the MPEG4 video stream. Reference numeral 601 denotes an imagecorresponding to one after MPEG4 encoding, 602 denotes a portionindicating a designated bit rate, 603 denotes a horizontal slide bar fordesignating which of the image quality and the frame rate is givenhigher priority, and 604 denotes a knob for indicating a settingposition of the current image quality and frame rate. The user can watchthe preview screen to operate the button for the user IF 109 so tochange each parameter setting. The control circuit 110 generates acontrol parameter for conversion and sends to the scaler 103 and theMPEG4 encoder 104. The MPEG4 encoder receives the information, reflectsthe above parameters and generates MPEG4 video stream in real time.

By configuring as described above, a decode image with the parametersbeing set is displayed in real time when the video stream is converted,so that the result can be obtained immediately. The user can check theresults of the parameters set by the user to interactively set theparameters to provide a desired image, and ease of use is improved.

When the user IF 109 is operated to decide and select a conversionparameter, the converted MPEG4 video stream is output from the terminal105 according to the selected conditions. The parameters of suchconditions are stored in a storage section provided in the systemcontroller 110. At the subsequent generation of the MPEG4 video scream,the system controller 110 controls the MPEG4 encoder 104 to encodeaccording to the stored parameters.

FIG. 7 is a diagram showing a portion to perform the stream conversioninto MPEG4 in this embodiment. Reference numeral 800 indicates a totalrecordable capacity of the storage medium 100 with time. And a period inwhich encoding and recording are conducted as MPEG2 video is indicatedby 801. A portion converted into the MPEG4 stream and output is a periodindicated by 802 and forms a part of the recorded portion. The start andend points of the above period are instructed by a conversion perioddesignation means possessed by the user IF 109. For example, they aredesignated by pushing a preview period decision button while reproducingthe MPEG2 video. Then, when preview output is designated by the user IF109, only the period 802 is reproduced by the MPEG2 decoder 102 toconvert into the MPEG4 stream. By configuring as described above, whenthe part with a short stream period in which transmission of motionpicture mail or the like can be made is reviewed for example, only theimage quality of a part to be actually sent can be checked.

FIG. 3 is a diagram showing an example of the image signal shown in thepreview screen of FIG. 2. Specifically, it shows the relation betweenthe MPEG2 stream reproduction position and the MPEG4 stream along thetime axis. For the MPEG2 stream recorded in the medium 100, the portionto be reproduced for preview is designated by the conversion perioddesignation means of the user IF 109 described with reference to FIG. 7.Thus, only a specified period of a short time, e.g., 10 or 20 seconds,is repeatedly reproduced. The MPEG4 encoder 104 successively encodes thestream of the designated period, and the display circuit 107 repeatedlygenerates the preview screen.

By configuring as described above, the user can check by repeatedlywatching the screen encoded according to the parameters determined bythe user. In other words, checking of the same design facilitates detailcomparison with attention paid to a given image or a given parameter.And, for example, when the motion as a design or a pattern of the objectis very small, the image quality tends to deteriorate at the time ofMPEG4 encoding. Even in such a case, adjustment of the parameter byrepeatedly reproducing allows to maintain the image quality of aportion, whose image quality is easily disturbed, up to a leveldesignated by the user. Accordingly, the adjustment can be madeaccording to the desire of the user to have the frame rate sacrificed tosome extent. Especially, when a stream of a short time such as motionpicture mail is generated, the parameter adjustment can be continuedwithout stopping the preview by repeatedly reproducing. Meanwhile, thestream is not output from the terminal 105 to the outside but outputafter the decision with the preview, so that unnecessary data beingadjusted is prevented from being received by the outside device.

For example, when MPEG2 video stream once recorded on a DVD is sent asMPEG4 video stream to another device according to this embodiment, acombination of the frame rate and the image quality most desired by theuser in the bit rate at the time of transmission can be previouslydetermined on the preview screen and sent.

When the scaler 103 is controlled according to the designated resolutionby the user IF 109 to change the resolution of the MPEG4 stream to beoutput, the user can interactively set on the preview screen. Besides,when a total size of streams to be output is limited, it can be outputfrom the terminal 105 after a total bit amount at the time of setting iscalculated on the preview screen and checked by the user whether thelimit is met.

FIG. 4 is a diagram showing another example of the image signal shown onthe preview screen of FIG. 2. In this case, it is configured that in anMPEG2 video stream 503 before the conversion into the MPEG4, only aparticular frame 505 is designated by the user IF. The MPEG2 streambeing reproduced repeatedly is encoded by the MPEG4, then only a frame506 in the MPEG4 stream corresponding to the particular frame 505 isoutput and shown on the preview screen. In this case, the MPEG4 streamis converted as a motion picture from the MPEG2 stream, but only a stillimage of a particular frame is shown on the preview screen (the section601 of FIG. 2).

Specifically, it can be implemented by the display circuit 107 of FIG. 1which updates the image 601 in the preview screen only when an effectiveframe required to be shown is specified and freezes the entire image atall other times.

Such a configuration is effective when importance is attached to not theframe rate but to the image quality of each of the respective images,and a change in image quality against the parameter change can bedetected with higher reliability by decoding and outputting a particularimage. And, the parameter can be adjusted without seeing the afterimagesof other frames. For example, it is particularly effective when detailresolution is important for a surveillance system or the like.

FIG. 5 is a block diagram showing a second embodiment of the videorecording/reproducing device containing the apparatus or videotranscoder for converting an encoding system of video stream accordingto an embodiment of the present invention. This device is configured torecord/reproduce the video stream encoded by the MPEG4 encoder 104to/from a second medium 211 via a medium IF 210 in addition to theconfiguration of the first embodiment. The MPEG4 video stream reproducedfrom the medium 211 is decoded by an MPEG4 decoder circuit 201 andoutput from an output terminal 203 to the display device through adisplay circuit 202. This configuration is also possible to perform thepreview operation before recording on the second medium 211 as describedin the first embodiment. At that time, the MPEG4 stream output from theterminal 105 is decoded by the MPEG4 decoder 201 and outputted as apreview screen by the display circuit 202.

This embodiment is effective when the MPEG2 stream recorded on the firstmedium 100 is converted into the MPEG4 video stream and backed up on thesecond medium 210. To generate a decode screen at the time ofpreviewing, a decode image may be read from the local decode framememory 106 and output to the preview screen via the display circuit 202in the same way as in the first embodiment.

FIG. 6 is a block diagram showing a third embodiment of the videorecording/reproducing device containing the apparatus or videotranscoder for converting an encoding system of video stream accordingto the present invention. This device outputs the converted MPEG4 streamfrom the terminal 105 and sends to an outside device 300 (e.g., ageneral-purpose personal computer (hereinafter called as the PC)). Thesent MPEG4 stream is decoded by a software decoder on the PC 300 andoutput from a terminal 310 so to display an image on an unshown displayor the like. At this time, the preview screen as described in the firstembodiment can be generated by software contained in the PC 300. Whenthe preview screen is checked from the PC to find that it is necessaryto change the parameter, only set parameter information is sent from thePC to the video recording/reproducing device as the main device. Morespecifically, a signal line is disposed to send to the control circuit110. This signal line can be, for example, control signal communicationsvia USB, communications by IEEE1394, wireless communications using awireless LAN, or the like.

Based on the parameter sent from the PC 300, the MPEG4 stream is changedand output to the PC 300, so that the stream conversion to the user'sdesired image quality can be made. In this embodiment, the conversionparameter setting can be changed by a remote control because the videorecording/reproducing device is separated from the outside device (PC)which sets a parameter for conversion. And, the device configuration canbe simplified because the main body does not have an MPEG4 decodedisplay circuit therein.

In the above-described embodiments, the change of video stream fromMPEG2 to MPEG4 was described, but the present invention can also beapplied to an image compression encoding in general. It is apparent thatthe present invention is also effective when used for other imageencoding systems, for example, H.263 and JPEG. It is also effective tochange a frame rate and a bit rate in the stream conversion between thesame encoding systems. For the preview screen configuration, even whenanother screen configuration is used, the same effects can be obtainedby a screen having a configuration based on a combination of the previewimage and the adjustment parameter. Besides, the storage medium is notparticularly limited, and an optical disk, a magnetic disk, an ICmemory, or the like can be used.

According to the present invention, the ease-of-use at the time ofconversion of a captured image or the like to the user's desired imagequality is improved.

The invention may be embodied in other specific forms without departingfrom the spirit or essential characteristics thereof. The presentembodiment is therefore to be considered in all respects as illustrativeand not restrictive, the scope of the invention being indicated by theappended claims rather than by the forgoing description and range ofequivalency of the claims are therefore intended to be embraced therein.

1. An apparatus for converting an encoding system of video stream whichconverts a first video stream recorded on a storage medium by a firstcompression encoding system into a second video stream using a secondcompression encoding system, comprising: a reproducer reproducing thefirst video stream recorded on the storage medium; a stream converterconverting the reproduced first video stream into the second videostream; a conversion period designator designating a conversion periodin the first video stream; a stream output unit outputting the convertedsecond video stream; and a preview output unit outputting an imagecorresponding to a decoded image of the converted second video stream toa display device; said stream output unit outputting the second videostream for a period designated by the conversion period designator.